Story Segmentation and Topic Detection in the Broadcast News Domain

In this paper we present algorithms for story segmentation and topic detection. Both algorithms are online algorithms and use a combination of machine learning, statistical natural language processing and information retrieval techniques. The story segmentation algorithm is a two stage algorithm that uses a decision tree based probabilistic model in the rst stage and incorporates aspects of our detection system via an information-retrieval based re nement scheme in the second stage. The topic detection algorithm is an incremental clustering algorithm that employs a novel dynamic clusterdependent similarity measure between documents and clusters. Cseg and topic-weighted Cdet for these algorithms on the 1998 TDT2 Evaluation are 0.1651 and 0.0042.